I consider novel ways by which Gamma-Ray Bursts (GRBs) and quasars can be used to probe the universe. Clues about how and when was the intergalactic medium ionized can be read off the UV emission spectrum of GRB explosions from the first generation of stars. The existence of intergalactic and galactic stars can be inferred from their gravitational microlensing effect on GRB afterglows. Prior to reionization, quasars should be surrounded by a halo of scattered Lyman-alpha radiation which probes the neutral intergalactic medium (IGM) around them. The situation is analogous to the appearance of a halo of scattered light around a street lamp which is embedded in a dense fog. Outflows from quasars magnetize the IGM at all redshifts. As a result, the shocks produced by converging flows during the formation of large scale structure in the IGM, accelerate electrons to relativistic energies and become visible in the radio regime through their synchrotron emission and in the gamma-ray regime through their inverse-Compton scattering of the microwave background photons. During transient episodes of strong mergers, X-ray clusters should therefore appear as extended radio or gamma-ray sources on the sky.